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A planning and discussion committee was formed at the 
University of Arkansas (Little Rock) to evaluate the role that 
faculty teaching assessment might take in a planned liberal arts 
teaching excellence project. The committee leader attended an 
Assessiiient Strategies Workshop conducted by Trudy Banta at the 
University of Tennessee, The committee head then shared materials 
from the workshop with all committee members and other faculty and 
had a research assistant check the availability of all 
bibliographical material, about half of which was readily available. 
The committee met continuously for project planning over the weeks 
following the workshop in order to prepare the project proposal for 
liberal arts teaching excellence. Though the committee*s proposal for 
further funding was denied, the discussion committee's work had many 
beneficial results including increased understanding by faculty 
participants of assessment and its role in higher education. The 
committee developed resource materials for assessment and identified 
library holdings , a draft for a faculty development project was 
produced and participants are positioned to plan future nrojects. 
Appended is a copy of the draft project proposal on faculty 
development, (JB) 
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PROJECT SUMMARY: Teaching Excellence in the Liberal Arts 

At the heart of this project was the attempt to build a 
QQ sound assessment component for a faculty development 

project. We found that, essentially, a rich and varied 
^ array of assessment strategies can be used to evaluate 



teaching and learning in the humanities, even though faculty 
members may be initially dubious on the validity of such 
methods. Long-range planning is probably necessary to 
commit faculty to what is, after all, a risky proposition — a 
redirecting of their approaches to teaching. 
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Teaching Excellence in the Liberal Arts 
University of Arkansas at Little Rock 
Little Rock, AR 72204 



Contact: Steve Anderson 

Department of English 
(501 ) 569-8312 

EXECUTIVE SUMMARY 

Project Overview 

This project grew out of my work in faculty 
development. Assessment was the missing link in the 
proposal for a large-scale teaching excel lence project that 
I had earlier submitted to FIPSE. The funding would allow 
me time and resources to supply the assessment component. 

Purpose 

It was, it seemed, especially important to present the 
concepts of assessment to a faculty that was either unaware 
of the "assessment revolution" that was taking place, or was 
hostile to the concept of assessment. The major program 
that I intended to submit to FIPSE called for involving 
large numbers of liberal arts faculty in projects in 
teaching excellence. Faculty would have to feel comfortable 
with assessment before engaging in the projects. 

Background and Origins 

Having spent my teaching career at institutions that 
enroll large percentages of no n- t r a di o na 1 students, I've 
long been dedicated to the methods of developinj^ quality 
education for students who must function under less than 
ideal circumstances. I worked from a number of important 
assessments written in the mid-eighties ( Involvement in 
Learning , Integrity in the Curriculum , a nd To Reclaim a 
Legacy ) to identify the traits considered important for 
higher education. 

Project Descrintion 

At the heart of my project was a committee that, would 
discuss the role that assessment would take in the project 
that I was planning. My background in the field was 
enhanced by attending the Assessment Strategies Workshop 
conducted by Professor Trudy Bant a (of the University of 
Tennessee) at Memphis, Tennessee, in October 1988. All the 
sessions I attended were most helpful, and the packet of 
assessment materials (including the b i':j 1 i og r ap hy ) was 
especially so* I made the packets of material available to 
all committee members--and other faculty--and had a research 
assistant check the availability of all bibliographical 
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entries (about half the material was readily available to 
us)* We also talked to a number of humanists over the 
country who were interested in participating in our faculty 
development project* 



The committee met continuously for 
over the weeks following the Assessment 
Even though the deadline for submission 
proposals to FIPSE allowed no more time 
still confident that we had assembled a 
excellence project that was supported 
component. In the return comments on 
learned that the assessment component 
sufficient to justify the project. I 
this judgment, but can only say that 



project planning 
Workshop I attended, 
of preliminary 
than that , I was 
viable teaching 
by a sound assessment 
the proposal , I 
was still not 
have no quarrel with 
we had less than an 



ideal amount 
s ubmi s s i o n . 



to time to shape the project for a 1987 



Even though the preliminary proposal had not gained the 
support of the reviewing committees, I chose still to submit 
a final proposal. I was told that this was permissable 
under the rules, and I thought I could address the 
deficiencies of the assessment model outlined in the 
preliminary proposal, especially since I'd had several weeks 
to grow more familiar with the nuances of assessment. This 
proposal, too, was turned down by FIPSE. Although we have 
been turned down to this point, I will continue to refine 
and develop the assessment component for this project. I 
believe I have a viable concept for developing teaching 
excellence; apparently only the assessment component is 
want i ng . 



However, the project involved significant alternate 
activities. For one, it served to broach the issue of 
assessment with administration and faculty. Many faculty 
know about assessment, if they about it at all, from 
journalistic sources. They have not looked closely at the 
research material in journals, which shows assessment in a 
positive light. 



Pro j e c t Re s u 1 1 s 

The project has had many benign results, even though we 
did not achieve the hoped for funding. Portions of our 
faculty has grown in its understanding of assessment and its 
role in higher education today. We have developed resource 
materials for assessment and have identified our library 
holdings. I have a draft of a faculty development 
proj ect--including a strategy on as s e s smen t-- t ha t I can work 
to modify and refine. And, finally, I am in a position to 
plan future projects with a strong sense of the need to 
account for assessment. 
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Summary and Conclusions 

I have learned that work in faculty development must 
necessarily begin in the assessment of teaching and 
learning, tasks that are especially difficult in humanities 
areas. Once faculty members understand what assessment 
is--or understand the rich diversity of strategies and 
methods included under the term assessment---they are rarely 
hostile to it. 
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FINAL REPORT: Teaching Excellence in the Liberal Arts 

Project Overview 

This project grew out of my work in faculty 
development • I have designed a number of projects that 
attempted to bring opportunties to faculty for developing 
and improving their teaching techniques, especially when 
dealing with non-traditional students. The purpose of this 
project was to consider ways to quantify improvements that 
we might achieve in teaching excellence. Given that I work 
in the liberal arts fields--areas that are consi-dered 
difficult to assess quanti ta tively-- this kind of research 
seemed particularly important. Assessment seemed to be the 
missing link in putting together a vi<ble proposal for a 
large-scale teaching excellence project. 

Assessment was the missing link in the proposal for a 
large-scale teaching excellence project that I had earlier 
submitted to FIPSE. The funding would allow me time and 
resources to supply the assessment component. 

Purpose 

It was, it seemed, especially important to present the 
concepts of assessment to a faculty that was either unaware 
of the ''assessment revolution'* that was taking place, or was 
hostile to the concept of assessment. (Many faculty see 
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assessment as having a political purpose, a way by which 
state legislatures could shake the semi-autonomy of the 
university system.) The major program that I intended to 
submit to FIPSE called for involving large numbers of 
liberal arts faculty in projects in teaching excellence. 
The key to determining whether teaching excellence could be 
quantified was to link the projects to assessment. Faculty 
would have to feel comfortable with assessment before 
engaging in the projects. 

Background and Origins 

Having spent my teaching career at institutions that 
enroll large percentages of non-tradional students, I've 
long been dedicated to the methods of developing quality 
education for students who must function under less than 
ideal circumstances. That is, how do we deliver quality 
education to students who have substantial r e s po ns i b i 1 i t e s 
to family and job? In many cases, f a cul t y-- t r a i ned in 
prestigious graduate i ns t i t u t i o ns--o f t e n do not feel 
comfortable teaching students having problems »!!asterlng even 
the basic skills. 

I worked from a number of important assessments coming 
out of the mid-eighties ( Involvement in Learning , Integrity 
in the Curriculum , and To Reclaim a Legacy ) to identify the 
traits considered important for higher education. The 
traits included encouraging students to become autonomous 



S 
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learners, to program learning through inquiry, to develop 
analytical and problem-solving skills, and to engage in the 
reading of original texts. We identified these as learning 
goals that professors might work to encorporate in their 
courses. FIPSE was originally interested in this project, 
but saw a need for the assessment component. That is, how 
does one measure a growth in problem-solving skills? or 
measure growth in becoming autonomous learners? 

Project Description 

At the heart of my project was a co mm it tee that would 
discuss the role that assessment would take in the project. 
For that I purpose, I believe the members of the committee 
were well chosen. Dr. Roby Robertson, of the Political 
Science Department, had done work in assesssraent in 
connection with public administration programs. Dr. Mark 
Krain, a sociologist, was familiar with the research 
literature on assessing social programs. Dr. Belinda 
Blevins, from the Department of Psychology, has her own 
unique background in assessment from her work in cognitive 
psychology. Dr. Thomas Kaiser, of the History Department, 
could bring to the committee the perceptions of the 
humanist, who often have not dealt with quantitative 
assessment. Dr. Kaiser and I were in much the same 
position, because I had done little work in quantitative 
assessment. 

ERLC 
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My background in the field was enhanced by attending 
the Assessment Strategies Workshop conducted by Professor 
Trudy Banta (of the University of Tennessee) at Memphis, 
Tennessee, in October 1988* All the sessions I attended 
were most helpful, and the packet of assessment materials 
(including the bibliography) was especially so. I made the 
packets of material available to all committee members — and 
other faculty — and had a research assistant check the 
availability of all bibliographical entries (about half the 
material was readily available to us)» 

One of the key tasks was to encorporate a strong 
assessment component in the project that we intended to 
submit to FIPSE. The committee met continuously for this 
purpose over the weeks following the Assessment Workshop I 
attended* Even though the deadline for submission of 
preliminary proposals to FIPSE allowed no more time than 
that, I was still confident that we had assembled a viable 
teaching excellence project that was supported by a sound 
assessment component* In the return comments on the 
proposal, I learned that the assessment component was still 
not sufficient to justify the projecu* I have no quarrel 
with this judgment, but can only say that we had less than 
an ideal amount to time to shape the project* 

Even though the preliminary proposal had not gained the 
support of the reviewing committees, I chose still to submit 
a final proposal. I was told that this was permissable 
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under the rules, and I thought I could address the 
deficiencies of the assessment model outlined in the 
preliminary proposal, especially since I'd had several weeks 
to grow more familiar with the nuances of assessment. 
Meanwhile, I had the opportunity to confer with humanists 
and scientists across the country who had engaged in 
teaching excellence projects. I thoughu that their ideas 
and their potential participation made the proposal 
especially strong. This proposal, tliough, was turned down 
by FIPSE. Although we have been turned down to this point, 
I will continue to refine and develop the assessment 
component for this project. I believe I have a viable 
concept for developing teaching excellence; apparently only 
the assessment component is wanting. 

However, the project involved significant alternate 
acitivities. For one, it served to broach the Issue of 
assessment with administration and faculty. Many faculty 
know about assessment, if they about it at all, from 
journalistic sources. They have not looked closely at the 
research material in journals, which shows assessment in a 
positive light. 

I should mention, too, the work of Dr. Roger Webb 
(Department of Psychology) in our project. As the 
assessment consultant, he advised us on ways of validly 
measuring student learning. As an example of his work, I 
have included his report concerning the activities of the 
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Assembly Committee on Evaluation* Although our project did 
not fund Dr. Webb for this work, our projects did run 
simultaneously. 

Further, the project has prompted us to pay more 
attention to the assessment issue and to provide resources 
for assessment practices. For long, the whole issue seems 
to have been limited to (Education departments, or other 
interests that involve pedagogy. We now know that the issue 
is large and urgent. 

Pro j ect Results 

As stated above, 1 believe the project has had many 
benign results, even though we did not achieve the hoped for 
funding. Portions of our faculty has grown in its 
understanding of assessment and its role in higher education 
today. We have developed resource materials for assessment 
and have identified our library holdings. 1 have a draft of 
a faculty development p r o j e c t-~ i nc lud i ng a strategy on 
assessment-- that 1 can work to modify and refine. And, 
finally, 1 am in a position to plan future projects with a 
strong sense of the need to account for assessment. 

The original proposal did not include a formal and 
extensive strategy for evaluating the project. 1 assumed 
that evaluation would lie acceptance or non-acceptance of 
the large proposal. Since the project was not accepted in 
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its current form, I assumed less than satisfactory results 
for that aspect of the project. 

In follow up discussions with faculty — informally 
conducted — I have found that the responses have been very 
positive. The members of the coraraittee were happy to have 
the opportunity to review assessment procedures, and felt 
that they had given the project their best effort. The 
graduate assistant, James Dinkins, was especially happy for 
the opportunity to do research on assessment. In fact, his 
research carried him into suject matter that he incorporated 
into his masters thesis. 

The project will continue under the form of submitted 
project proposals. Although as grant director, I was unable 
to resubmit in October 1988 — I was on Off-Campus Duty 
As signmen t~~I will submit a faculty development project in 
1989. 

Summary and Conclusions 

I have learned that work in faculty development must 
necessarily begin in the assessment of teaching and 
learning, tasks that are especially difficult in humanities 
areas. Faculty often have a natural antipathy toward 
measuring anything that it believes cannot be measured. So, 
as much as anything, we must learn how to approach faculty. 
Once faculty members understand what assessment is--or 
understand the rich diversity of strategies and methods 
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included under the term assessmenn--they are rarely hostzile 
to it. 
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DRAFT REPORT ' 
Assembly Committee on Evaluation 

November 12, 1987 
(revised January 12, 1968) 



A. General proble'^ 

The charge to the fvaluation Comnittes trom the President o^ the UALR 
UALR^s un;" h'T'°' "'''^'^ evaluating the eHectivene oi 

tive daU n ^h /"'"'"'"r ^^^^^^^ '''' ^= '^-^loP objec- 

aslno ? ! ^ performance 0^ undergraduate students and graduatei 
using the statement oi educational goals developed by the BRC (Blue Rib- 
bon Committee) as guidelines. That is, we need to be able to 
demonstrate 1) how well recent UALR graduates perform on the tasks de- 

ZtMl III '''' ^''^^"^ Irr...... reform: a ed 

for by the BRC improve existing performance levels? We need to be clear 
on one point here: our goal is to measure institutional effectiveness 
and not individual student achievement. ttiveness 

There is probably some urgency to make substantive progress in this 
area. Such an evaluation will be an important tool for institutional 

m vement. n addition, educational agencies, leqislators n e 
puDl c generally see. to be m a mood to demand some evidence of e feet 
for the dollars spent on higher education. Unfortunately, 11- on .ived 
evaluation programs are more often destructive than consJ uc ve UALR 
needs to have a defendable evaluation program in pla f 'h a 

program is mandated by an outside agency. 

lllll considerable complexity. If we want to talk 
about levels of performance of UALR students on the educational goals 
developed by the BRC, and then, hopefully, be able to talk about im- 
prove.en in those levels of performance, we must meet two conditions: 
we must have measurinn i n = f F-.im=,r,f ^ i.._u-- ., . . . 



rp-fi f "*^^5uring instruments and techniques that accurately 

reflect the goals, and we must have samples of students that are repre- 
sentative of the group of students we want to describe, 



It IS difficult for school systems and institutions to come to grips 
with how seriously measuring instruments and sampling techniques pffpct 

thnn?" -^'^ ^'^^ ^"earch on educational ouUc^es. 

.cnool systems, for example, routinely report high average SAT scorps 
witn pride as an index of successful education. This is misleading 
statistic for two reasons. The SAT test is an individual aptitude test 
reflecting the inoorn talent and cultural background of the student'much 
more than any contribution of the school system, and the students taking 
the test are sel f -sel ected volunteers. The College Board and most 
sophisticated test users know that high average SAT scores is primarily 
inaicative of a low percentage of students taking the test. 

It is the intention of this committee to plan our evaluation efforts in 
such a wav that will allow UALR to make valid statements about the 
achievement levels of UALR students and graduates. We do not intend to 

. i ^ 

1 r 
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waste our ti^e and the University's .oney collecting data for political 

Z lT^ll\r'°'''- ^° systematic evaluation, we n ed o 

do so wit the sa.e degree rigor that we would approach rese'ch we 
do ior publication in scholarly journals. rese^. cn we 

eHoTortLlH' P^oble^s facing an evaluation 

effort of the sort we contemplate here: measurement instruments and 
sampling technique. We propose to attack both in parallelTshion 

B. Measuring instruments: 

^^Lrl'^J'^ °^ instruments is critical. The committee believes that 
there must be some use of nationally standardized tests. We Med o e 
able 0 compare our students to national norms, and we will noibe 
credible without some data based on independen yards ic s A the -^ame 

ture ca cu ations on that decision. It is unlikely that we will find a 

"qu r ALR,'"'d': 'J^' completely reflects the BRC goal^ (lha ar 
unique to UhLR) , and tests can become the "tail that wags thp doq" It 
IS shocking for example, to realize the extent to which the'con ents of 
standardized achievement tests determines the curriculum o5 our Jub^ic 

IJon^'J''" J'/'"'" °^ '''' materials available on a na- 

tional basis that touch to some extent the content areas t a w need to 

Ht I'n, r ^n--" '''' ''''''''' ""^^ program lom h 

AuT, and new program called the Academic Profile, also from ETS Each 
has Its strengths and problems. 

il n^^f CLEP tests are designed to allow individual students to 
demons rate knowledge equivalent to general education requirements and 
to articular courses. One or more of the area exams .ight be u ef- 
ri.ing junior exams, but they are not very close to the BRC objectives ^ 
Experience wit the CLEP tests suggests that they tend to be ! 
telligence tests, rather than an achievement tests, and that scores are 

Jfm" C E " sU ^"^^^^^ °^ education ro^ 

gram. CLEP tests require a minimum of 90 minutes and are the most ex- 

ensive of the sUndardized tests. Standard-setting exams for Jhe C^EP 
tests are probably available at little or no direct cost. 

2. The COMP program (College Outcomes Measures Program) from the ACT 
at empts to test areas that are close to those described by the BR^. 
These are communicating, solving proolems, clarifying values, function- 

r ?h"? "-ng -^-ce and t echnol ogy ' and 

he a ts. The logistics of giving and scoring the tests appears dif- 
ficult, thoug more study on that area is required. Costs are moderate. 

le m h'. I' '''' '''' = ^^'^"Ity scoring 

teams that will, spend a total of about 60 minutes on each written and 
ora response. A two hour objective version is available that reouirps 
no faculty time, but is of questionable validity. 

3. The Academic Profile program from ETS is new and relatively un- 
tested t IS designed to measure college level reading, writing, crit- 
ical thinking and math skills in the areas of the humanizes, social 
sciences, and natural sciences. The Academic Profile would cover many, 



though not all, of the BRC objectives. One hour test forms allow admin- 
istration in a class period. The program is designed to assess institu- 
tions and programs, rather than individuals. A relatively complex samp- 
ling system IS required to test groups of students on three tests that 
can be added together to obtain a measure of the program. Statements 
about individual students are not available unless the three hour long 
form 15 used. Costs are reasonable during this year (considered a pilot 
year), but will be higher in the future. 

It is the recommendation of the Committee that we exnlore all thrgp nf 
these testing programs and p ossibl y do some oilot testing with .11 f hron 
to asse ss their usefulness f.jr IJALR. ^ ~" 

nLJLrr'I'" standardized tests that are available, it will be im- 

portant to keep in mind aspects of the BRC goals that are missed by each 
test. We will then have to cast about for other measuring devires that 
niigh be available. For example, there are published tests on critical 
thinking that need to be explored. Experts in the English Department 
need to be consulted in the area of writing assessment, which is a spe- 
cialized field. If we want to be comprehensive, we will certainly need 
to develop some of our own measuring devices. 

In addition to objective, standardized tests, we need to develop less 
formal, more open-ended methods for assessing student orogress. For e.- 
stuien.i'''' ' Po^ibHity of keeping a writing folder on individual 
students covering everything from freshman composition to their last 
senior courses. Such records would allow us an opportunity to put some 

si? sf^i % %H psychologically 
sat sfying to those a,T,ong us who are mistrustful of numbers. Develop- 
ment of these measures will tax our creativity. 



C. Sampling problems: 

The discussion of tests and measurement begs the more critical question 
of test who? Simply collecting test data on students does not allow 
us 0 say anything about UALR's programs. We need to sample in some 
systematic fashion so the universe of students to which our inferences 
apply can be t.. early delineated. The importance of the sampling problem 
cannot be overstated. It would appear that ,T,ost educational outcomes 
research suffers in this area. It is possible, for example, for a 
school to significantly increase (or decrease) scores on standardized 
tests by redistricting, increasing admissions standards, recruiting from 
different groups, etc. without doing anything different in its rur- 
ri cul um. 

Except for some possible pilot testing to check instruments, the Com- 
mittee does not want to start systematically testing students until we 
nave the sampling problem in hand. 

There are two major evaluation goals: 

I. Are the students we are presently qrad,,ati-.G adc-cjuitely pr^oarc-d 
using the guidelines of the BRC as a standard" ' 



2. As we implement the changes recommended by the BRC. doP. fhp 
formance our students improve? ' ^ P^^" 

SALrHn^Jil'' "'."r^ '° ''^'"'^ °^ the degree to which 

is'a rch^:^:cH:;\v:-;em^^ ^^"-^^-"^^ .uLt.on-:srJh.t 

to or control over people once they gradua^n w^'^ssil a^ tJ" 
our first target for assessment i s uL seniors If 1 r. . """"" 
reasonably objective measures nn fhi= H we can get some 

vPAr.; fkl "^^'^'^'^^''^ measures on this group over a period of several 

progress (e.g. as students comolete the core curricuJul) ■ 

I t' se h:^-' addressed in many of our discussions is o l- 

"aipJ^e: u :r?ardrjvTa:,prer:nL:rrn°i;,rh 

antee that the classes adequately r'f ected t " H^l S % h''/! '° 
would have the problem of class cuU^Js ' 




Mill insure r Pr .^^^rnt. Jt l nn nf -11 "" 

If we have these students to test it- i i h ^ . . 

to make clear to students that wl I'nportant to reme.,ber, and 

gram, not the student! ' assessing the undergraduate pro- 

UALR If i= n= 11 diiow Us to trai.k progress over time at 

field t at orLp'f ' ^r'r''' '"'"^ ''''''' the experimental design 
solie For' h ''''''' ""^^ ^'^^^ P^^l^l^^' than they ' 

where their abilities are coming frol ?lte" ^"^^^ ^'^^^ 



D. 



A long range testing pr 



ogram 
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E. Cost estimates: 

$1^0 OOfi anH 4-h,4. ,_, annual budget o-f around 

*UiO,o00, and that is probably not an unreasonable target Hqure for thP 

and scoring (at *6.50/test) wiH b ^ 9.0 ' p f t". °' /^terials 
prograr, would have approximate v *L " "^'"^ '''^^ 

tioJ" ^S^d^' "ithout budgeting .„y ,dd,- 



::£!i£;:"i;r"r '-<;s j a-::^2/::a::r;.,:"f ^ 



Members of the committ 



ee : 



RSbeit'er'adl""" I","'''' ^'"'^ ^"^^^1° 

Floyd Martin Eric Melvin Panye Porter 

Jennifer Rector Roger Webb (chair) ^ 
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